Probabilistic Cache Filtering

Distinguishing transient blocks from frequently used blocks enables servicing references to transient blocks from a small highly-associative auxiliary cache structure. By inserting only frequently used blocks into the main cache structure, we can reduce the number of collision misses, thus achieving higher performance and allowing the use of direct mapped caches which offer lower power consumption and lower access latencies. In this paper we introduce a simple probabilistic filtering mechanism to identify and select the frequently used blocks. We show that a 16K direct-mapped L1 cache, augmented with a fully-associative 4K filter, achieves on average 11% more instructions per cycle than a regular 16K, 4-way set-associative cache, and even 4% higher IPC that a 32K, 4-way cache, while consuming about 60–70% less dynamic power than either of them.